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Abstract 

A mixing Markov chain is proved to be realized as a random walk in a directed 
graph subject to a synchronizing road coloring. The result ensures existence of 
i appropriate random mappings in Propp-Wilson's coupling from the past. The proof 

is based on the road coloring theorem. A necessary and sufficient condition for 
approximate preservation of entropies is also given. 
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'• 1 Introduction 

S: 

Our purpose is to realize a mixing Markov chain as a suitable random walk in a directed 
CN ' graph, which is generated by a sequence of independent, identically distributed {IID for 
^ ■ short) random variables taking values in the set of mappings of the state space. 

m 
o 

o , 

2 ' Let be a set of finite symbols, say V = {1, . . . ,fn}. Let Y = {Yk)kei be a Markov 
chain taking values in V and indexed by Z, the set of all integers. We call Y stationary 

if {Yk+n)k& = iYk)k& for all n G Z. We write Q = {qx,y)x,y&v for the one-step transition 
probability matrix of Y , i.e.. 



X 
J3 



1.1 Notations 



q,^y = P{Y^=y\Yo = x), x,y&V. (1.1) 

The n-th transition probability matrix is given by the n-th product = {qx,y)x,yev- We 
call Y irreducible if for any x,y &V there exists a positive number n = n{x, y) such that 
1x,y > 0- We call Y aperiodic if the greatest common divisor among {n > 1 : q^.^. > 0} 
is one for all x & V. We call Y mixing if Y is both irreducible and aperiodic, which is 
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equivalent to the condition that there exists a positive integer r such that q'^y > for all 
x,y E V. 

Let S denote the set of all mappings from V to itself. For (Xi, (72 € S and x E V, we 
write o"2cria; simply for cr2(c"i(a^))- 

Definition 1.1. Let Q be the one-step transition probability matrix of a stationary 
Markov chain. A probability law yU on S is called a mapping law for Q if 



qx,y= Yl ^'l/^'^- 



:i.2) 



Definition 1.2. For a probability law /i on S, a ^-random walk is a Markov chain 
{X, N) = (Xfc, Nk)k& taking values in F x S such that = {Nk)k^i is IID with common 
law n such that each A^^ is independent of a {Xj , Nj : j < k — 1) and 



Xfc = A^fcXfc_i a.s. for A; G Z. 



;i.3) 



Let y = (Yk)k(zi be a stationary Markov chain with one-step transition probability 

matrix Q. Let {X, N) be a /i-random walk. Then it is obvious that Y = X if and only 
if /i is a mapping law for Q. For any stationary Markov chain Y, we can find a mapping 
law /i for Q (see Lemma [XT]) . 



Let us illustrate our notations. See Figure 1 below, where V = {1,2,3} and 

<?!,! qi,2 qi,3] r 2/3 1/3 

Q = ^2,1 q2,2 '?2,3 = 1/3 2/3 

53,1 93,2 g3,3 2/3 1/3 



;i.4) 



Let a^^\ a^'^\ o"*-^^ and a^'^^ be elements of S characterized by Figures 2, 3, 4 and 5 below, 
respectively. The transition probability Q possesses several mapping laws; among others, 
we have /i*^^-* and /i*^^^ defined as follows: 



2/3, 



1/3, 
1/3. 



;i.5) 
;i.6) 



The two random walks (X, N) corresponding to /i'-^^ and /i*^^-* have distinct joint laws, 
but have identical marginal law of X which is a Markov chain with one-step transition 
probability Q. 




Figure 1. Transition probability 
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1.2 Realization of mixing Markov chain as a /i-random walk 

Our aim is to choose a mapping law fi which satisfies a nice property. 

Definition 1.3. A subset Sq of S is called synchronizing if there exists a sequence s = 
(o"p, . . . , o"i) of elements of Sq such that the composition product (s) := ap ■ ■ ■ ai maps V 
onto a singleton. 

First one of our main theorems is the following. 

Theorem 1.4. Suppose that Y = (Yk)kez is mixing. Then one can choose a mapping law 
for Q so that /i has synchronizing support. 

Theorem 11.41 will be proved in Section |3l 

Let us explain how our //-random walk is related to road coloring. The support of /i, 
which we denote by {a^^\ . . . , a^"^^}, induces the adjacency matrix A of a directed graph 
{V,A) which is of constant outdegree, i.e., from every site there are d roads laid. Then 
each element a^^^ , • • • , o"*-"'^ may be regarded as a road color so that no two roads from the 
same site have the same color. For a //-random walk {X, N), the process X moves in the 
directed graph {V, A) being driven by the randomly-chosen road colors indicated by N 
via equation (11.31) . Thus we may call {X,N) a random walk in a directed graph subject 
to a road coloring. For example, the directed graphs induced by /i^^^ and /i*^^-* which are 
defined in (11. 5p and (II. 6p . respectively, are illustrated as Figures 6 and 7, respectively. 
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Since a^^^a^'^^V = {3}, we see that the support of fx^^^ is synchronizing, while we can easily 
see that the support of /i*^^-* is non- synchronizing. 




Figure 6. The graph induced by /i^^^ Figure 7. The graph induced by //^^^ 

Let us come back to the general discussion. If (X, A^) is a /i-random walk and if the 
support of /i is synchronizing, then the process X may be represented as 

Xk = F{Nk,N,_i,...), keZ (1.7) 

for some measurable function F : — )■ V. In fact, define 

T{Nj ■.j<k)= max{/ G Z, I < k : NtNk-i ■ ■ ■ NiV is a singleton}, (1.8) 

where we follow the convention that max = — oo. Since the support of /z is synchronizing, 
it holds that T{Nk, Nk_i, . . .) is finite a.s. for all G Z, so that we may define 

Xk = NkNk-i ■ ■ ■ NTiN,:j<k)Xo, keZ (1.9) 

for a fixed element Xq & V, but the resulting random walk does not depend on the choice 
of Xq. This is such a representation as f ll.7p . 

Letting /c = in the identity fll.9p . we have 

Xo = iVoX_i ■ ■ ■ NTiN,:j<0)Xo. (1.10) 

This shows that the stationary law of the Markov chain may be simulated exactly from an 
IID sequence. This method was a central idea of Propp-Wilson's coupling from the past 
(|13]: see also [9]). Our Theorem 11.41 assures theoretically that for any mixing Markov 
chain there always exists an appropriate mapping law such that Propp-Wilson's algorithm 
terminates almost surely. 

For the study of /^-random walks in the case of non- synchronizing supports, see Yano 
[T7]. Equation (11. Sp is called Tsirelson's equation in discrete time; see Yor [20j, Akahori- 
Uenishi-Yano [3] , Yano-Takahashi [18] , Yano- Yor [19] and Hirayama-Yano |10[ ITT] for 
the details. 

The representation Y = X = F{N) of Y by an IID sequence N of the form (11. 7p is 
called a non- anticipating representation. Rosenblatt ([H] and [15]) obtained a necessary 
and sufficient condition for a Markov chain with countable state space to have a non- 
anticipating representation Y = X = F{N) where N = {Nk)k& is an IID with uniform 
law on [0, 1]. 
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1.3 Condition for approximate preservation of entropies 



Let Y and (X, A^) as in Theorem 11.41 Let us compare the amounts of information of 
Y and in terms of their entropies. See the standard textbook [4J for basic theory of 
entropies. Let A be the stationary law of Y and define 

hiY) = - ^ K^)qx,y^ogq^^y (1.11) 

x,y&V 

and 

Miv) = -5^/i(^)iogM^)- (1-12) 

Since Y = X and X is a measurable function of N as in (11. 9p . we have 

h{Y) < h{N). (1.13) 

Note that Ornstein-Friedman's theorem (p^ and |H]) asserts that two mixing Markov 
chains which have common entropy are isomorphic. By this theorem, we see that, if 
equality holds in (ll.lSp . then Y is isomorphic to N. We do not have any general criterion 
on Y for existence of a mapping law such that Y is isomorphic to N. We will give an 
example for non-existence in Section [51 

We are interested in when we can take mapping laws such that the h{N) approximates 
the h{Y). Following Rosenblatt [13], we introduce the following: 

Definition 1.5. A stationary Markov chain Y is called p-uniform if there exist a proba- 
bility law u on V and a family {r^. : x G V} of permutations of V such that 

qx,y = Jy{rx{y)), x,yeV. (1.14) 

The second one of our main theorems is the following. 

Theorem 1.6. Let Y be a mixing Markov chain. Then the following assertions are 
equivalent: 

(i) There exists a sequence {/i^"-* : n = 1,2, . . .} of mapping laws for Q with synchro- 
nizing support such that the N^"'^ corresponding to /i*^"^ satisfies 

h{N^''^) ^ h{Y) asn^oo. (1.15) 

(a) Y is p-uniform. 

In particular, if h{N) = h{Y) holds for N corresponding to some mapping law fi for Q 
with synchronizing support, then Y is necessarily p-uniform. 

Theorem 11.61 will be proved in Section HJ 

This paper is organized as follows. In Section |2l we introduce several notations to state 
the road coloring problem. Sections [3] and H] are devoted to the proofs of Theorems 11.41 
and respectively. In Section we give an example for Theorem 11.61 
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2 Road colorings of a directed graph 



Let A = [A{y, x)]y^xev be a x V"- matrix whose entries are non-negative integers. The pair 
( V, A) may be called a directed graph, where, for x,y E V, the value A{y, x) is regarded 
as the number of directed edges from x to y. The set V is called the set of vertices and 
the matrix A is called the adjacency matrix. 

The graph (V, A) is called of constant outdegree if there exists a constant d such that 

^ A{y, x)=d for all xeV. (2.1) 

y(^V 

In this case {V, A) is called d-out. The graph (V, A) is called strongly connected if, for any 
x,y eV, there exists a positive integer n = n{x,y) such that A^{y,x) > 1. The graph 
{V, A) is called aperiodic if the greatest common divisor among {n > 1 : A^{x, x) > 1} is 
one for all x (zV . Note that (V, A) is both strongly connected and aperiodic if and only 
if there exists a positive integer r such that A^{y,x) > 1 for all x,y E V. We say that 
the graph (V, A) or the adjacency matrix A satisfies the assumption (A) if (V, A) is of 
constant outdegree, strongly connected and aperiodic. 

Recall that S is the set of all mappings from V to itself. For ai, (72 G S and x E V, we 
write a2crix simply for a2{<7i{x)). The set E acts V in the following sense: 

(cri(T2)x =ai{a2x), 0-1,(72 G S, x G V. (2.2) 

The set V = {1, . . . , m} may be identified with the set of standard basis {ei, . . . , Cm} 
of R"*. An element cr G S may be identified with the 1-out adjacency matrix a = 
Wiy^ ^)]y,xev given as 

a = [aci ■ ■ ■ acm] ■ (2.3) 
Under these identifications, we see that, for all x,y eV, 

a{y, x) = 1 if and only if y = ax. (2.4) 

Let (V, A) be a d-ont directed graph. A family {a^^\ . . . , a^^^} of elements of S (possibly 
with repeated elements) is called a road coloring of (V, A) if 

A = a« + --- + a(^). (2.5) 

Each (t(*) is called a roac? co/or. Note that there exists at least one road coloring of 
{V,A). Conversely, if we are given a family {a^^\ . . . , a^"^^} of elements of S (possibly 
with repeated elements), then it induces a unique d-out directed graph (y,A) given as 

Let So be a subset of S. A sequence s = (o"p, . . . , cr2, cti) of elements of Sq is called a 
T,Q-word. For a Eg-word s = (up, . . . , (125 cti); we write (s) for the product dp • • ■ o-20"i. The 
following definition is a slight modification of Definition 11.31 
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Definition 2.1. A road coloring Sq = {o'^^\ ■ ■ ■ ,o'^'^^} is called synchronizing if So as a 
subset of S is synchronizing. 



By this definition, we see that a road coloring Sq = {o'^^\ • • • , a^"^^} is synchronizing if 
and only if (s) \^ is a singleton for some So-word s. If we express 

s=(cr«^')),...,a«2»,a«i») (2.6) 

with some numbers i{l) , . . . , i{p) G {1, . . . ,d}, the assertion "(s) ^ is a singleton" may 
be stated in other words as follows: Those who walk in the graph (V, A) according to the 
colors this order will lead to a common vertex, no matter where they 

started from. 

Now we state the road coloring theorem. 

Theorem 2.2 (Trahtman ([IS])). Suppose that the directed graph {V,A) satisfies the 
assumption (A). Then there exists a synchronizing road coloring of{V,A). 

This was first conjectured in the case of no multiple directed edges by Adler-Goodwyn- 
Weiss [IJ (see also [2J) in the context of the isomorphism problem of symbolic dynamics 
with common topological entropy. For related studies before Trahtman [TB], see [7], [Hj 
and [5]. 



3 Construction of a mapping law on a synchronizing 
road coloring 

We need the following lemma. 

Lemma 3.1. Let Y be a stationary Markov chain with one-step transition probability 
matrix Q. Then there exists a mapping law fi for Q. 



Proof. First, we suppose that qx,y is a rational number for all x,y E V. Then we may 
take an integer d sufficiently large so that A{y,x) := qx,yd is an integer for all x,y E V. 
Then A := [A{y, x)]x^y,^v is the adjacency matrix of a d-out directed graph (V, A); in fact. 

My, x) = dY^ qx,y = d. (3.1) 

yeV y&V 

Let {cr*^^\ . . . , cr^"'^} be a road coloring of {V^ A) and define 

M^) = ^tl({* = l,---,^:cr« = ^}) (3.2) 

where tl(-) denotes the number of elements of the set indicated. Thus, for any x,y eV, 
we see that 

'"(^) = -M' = 1, . . . , : f^^^Hy, a;) = 1}) = -A{y, x) = q.^y, (3.3) 



7 



which shows that jj is a mapping law for Q. 

Second, we consider the general case. Let us take a sequence {Q^"^^ : = 1, 2, . . .} of 
one-step transition probability matrices such that is a rational number for all n and 
x,y G V and that — )■ q^^y as n — )■ oo for all x,y G V. Then for any n there exists 
a mapping law jj,^"'^ for Q^"-*- Since S is a finite set, we can choose some subsequence 
|^(n(fe)) ; = 1^2, . . .} and some probability law on E such that n^"-^^^\a) — )■ n{a) as 
A; — )■ oo. This shows that is a mapping law for Q. The proof is now complete. □ 



Now we proceed to prove Theorem 11.41 



Proof of Theorem Let Q = {qx.y)x,yev be the one-step transition probability matrix 
for a mixing Markov chain Y. 

First, we take an adjacency matrix A which is of constant outdegree such that 

AmI^' (3.4) 
[=0 if = 0. 

For this, we introduce a subset V x V defined by 

E = {ix,y)eVxV:qx,y>0}. (3.5) 

For each x G V^, we define the outdegree of at x by 

d{x) = Uix,y)^E:yeV} (3.6) 

and write d = max^-gy d{x) for the maximum outdegree of E. For each x & V, we may 
choose a site a{x) E V so that {x, (j{x)) G E. Now we may set 

{d — d(x) + 1 if y = cr(x), 
1 if y ^a{x) and {x,y) EE, (3.7) 

otherwise. 

Then this {A{y,x))x,yev is as desired. 

Since y is a mixing Markov chain, there exists a positive integer r such that q^y > 
for all x,y E V. Hence we have A'^{y,x) > 1 for all x,y G V; in fact, there exists a 
path X = Xo,Xi, . . . , Xn = y such that qx^-i^x^ > for A; = 1, 2, . . . , n, which implies that 
A{,Xk, Xk-i) > 1 for /c = 1, 2, . . . , n. Thus we see that (V, A) satisfies the assumption (A). 
Thus we may apply Theorem 12.21 to obtain a synchronizing road coloring {(t^^\ . . . , o"'-'^-'} 
of {V,A). Define 

K^) = -J{{i = ^.---.d: a« = d}), a G S (3.8) 



and define 



a(^Ti:y=crx 



8 



Then /i is a mapping law for Q and has synchronizing support. We also note that 

%,y = Q ii{x,y)iE. (3.10) 

Let 

e = min{g^,y : (x, y) e E} > 0. (3.11) 
If £ = 1, then we have Q = Q, so that /I is as desired. Let us assume that e < 1. Define 

g(^) = ^ (g - ^g) . (3.12) 

Then g(^) = {qi] l)x,y£V is a one-step transition probability matrix of a stationary Markov 
chain. In fact, we see that 

(1 - ^)(lxl = (lx,y - eqx,y > qx,y - el{(x,y)€E} > 0, x,y (3.13) 

and that 

yeV \y£V yeV / 

Now we apply Lemma [3. II to obtain a mapping law fx^^^ for g*^"^-*. Define 

/i = (1 - £)/i(') + e/I. (3.15) 
Since /x^^-* has synchronizing support, so does fi. For x,y gV, we have 

E M^)=(i-^) E E /^(^)- (3-16) 

= (1 - + ^Qx,y = qx,y, (3.17) 

which shows that yU is a mapping law for Q. The proof is now complete. □ 

4 Approximate preservation of entropies 



Let us prove Theorem 11.61 

Proof of Theorem M.bX Let us prove that (i) implies (ii). Note that 

KY) = - E Kx)(lx,y^^^Qx,y, (4.1) 

x^yaV 

h{N^-^) = - E/^^"H^)log/x("V). (4.2) 
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Taking a subsequence if necessary, we may assume that there exists a probabihty law /i 
on S such that ^^'^\a) — )■ for all a G E. Note that /i is a mapping law for Q but 
does not necessarily have synchronizing support. By the assumption fll.lSp . we see that 



hiY) = lim /i(Ar(")) = - y^(a) log /i((T). (4.3) 

n— >oo ' ^ 

o-GS 

For x,y & V , we set 

S(y,x) = {aGS:y = aa:}, (4.4) 

so that we have 

ctgi;(j/,x) 

Hence we have 

/w(cr) < g^;,/ whenever cr G E(?/,x). (4.6) 
Since t h- log t is increasing, we have 

- X] /^(^)log/^(^) ^ - yU(0-)l0ggx,2y = -gx,2;loggx,2;- (4.7) 

CTeS(j/,a::) o-gS(j/,x) 

Since Uyev ^(^Z; 2;) = S, we have 

/i(r) = - ^ 12(a) log 12(a) > q(x) for all xEV, (4.8) 

J/eV aeT.{y,x) 



where we set 



= -Yl ^^'y (lx,y, X eV. (4.9) 

y&V 



We take x G V such that 



q(x) = maxg(x). (4-10) 



Using (USD and fH?T]) . we have 

g(x) < /i(F) = ^ A(x)g(x) < q(x). (4.11) 



Thus we see that the equalities hold in (14. lip and that q(x) = q(x) for all x E V. For any 
X G V^, we combine h(N) = q(x) together with (14. 7p and then obtain 

- X] Ai(f^) log/^(cr) = -gx,?;loggx,?;, x,yeV. (4.12) 

(TeS(y,z) 
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Combining this with f l4.6p . we obtain 

/i(cr) = qx,y whenever a E T,{y,x). (4-13) 

Let Xq E V he fixed and let x E V. Since x) : y E V} is a partition of S, we may 

choose a permutation of V so that 

S(r,(|/),x)nS(y,Xo)^0, yEV. (4.14) 

This shows that 

(lx,T^(y) = (lxo,y, x,y EV, (4.15) 

which imphes p-uniformity of Y. The proof is now complete. 

Let us prove that (ii) implies (i). Let {xi, . . . , Xd} be an enumeration of the support of 
the law u in (11.141) . For i = 1, . . . ,d, we define 

a«(y,x) = l{.,(,)=.j. (4.16) 

For each x E V, there exists a unique yEV such that a^'^\y,x) = 1, so that we have 
(t(*) G S. By (fTTil) . we obtain 

gx,?; = a;,i/el^. (4.17) 

2=1 

Let A he as in (13.41) and let Si be a synchronizing subset corresponding to some syn- 
chronizing road coloring of (V, A). For sufficiently large integer n, we define a probability 
law yU*-"^ on S by 

^ i^} H- (4.18) 

Then it is obvious that /x^"-* is a mapping law for Q and has synchronizing support. 
Let us verify the condition (ll.lSp . On one hand, we have 

d 

z/(xj) log z/(xj). (4.19) 

n— ^-oo ' ^ 
i=l 

On the other hand, we have 

h{Y)=- J2 A(x)g,,,logg,,, (4.20) 

x,yGV 

d 

= - Yl A(a;)^a»(2/,x)z/(a:.)logz/(a;.) (4.21) 

x,y€V i=l 

i=l la;,yeV j 
d 

= -^z/(xi)logz/(xi). (4.23) 

i=l 



11 



This shows fll.151) . The proof is now complete. 

5 An example 

Let V = {1, 2}. Then E = {(12), (21), (11), (22)} where 

1 i 



□ 



z,j = l,2. 



(5.1) 



[2 ^ J_ 

Let < p < 1 and consider a Markov chain Y with one-step transition probabihty given 
by 

(5.2) 





?1,2 




P 1 — p 


.92,1 


'?2,2_ 




1 — p p 



Then it is obvious that F is a mixing Markov chain. Since 







'?2,2 




P 


^2,1 




Ql,2 




1 — p 



(5.3) 



we see that Y is p-uniform. 

It is obvious that the stationary law is given as 

A(l) = A(2) = 



(5.4) 
(5.5) 



We now see that 

h{Y) = ^{p) + ^{1 - p) 
where (p{t) = — tlogt. 

If /i is a mapping law for Q, then we have 

^(12) +/i(ll) =p, ^(21) +^(11) = 1 -p. (5.6) 

From this, we see that there exists some e with < e < min{p, 1 — p} such that 

£ = ;x(ll) = /i(22), /i(12)=p-£, fi{21) = l-p-e. (5.7) 

Conversely, for any e with < e < min{p, 1 — p}, we may define n = /x*-^^ by equation 
05. 7p so that fx^^^ is a mapping law for Q. 

If /i*^^) has synchronizing support, e should be positive. Let {X^^\ N^^^} be the fi^^^- 
random walk. We then see that 



h{N^''^) = 2^{e) + ^{p -e) + ^{l-p-e). 



(5.8) 



If p = 1/2, we see that h{Y) = h{N^^I'^^). 

Suppose that p ^ 1/2. Then, by an easy computation, we may see that 

hiY) < h{N^''>) (5.9) 
for all e with < e < min{p, 1 — p}. However, it holds that h{N^'^^) — )■ h(Y) as £ — ?■ 0+. 
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